Skip to content

feat(gax): implement resumable upload session resumption - #14677

Open
whowes wants to merge 1 commit into
whowes/resumable-upload-resume-wirefrom
whowes/resumable-upload-resume
Open

whowes wants to merge 1 commit into
whowes/resumable-upload-resume-wirefrom
whowes/resumable-upload-resume

Conversation

@whowes

@whowes whowes commented Oct 16, 2026

Copy link
Copy Markdown
Contributor

Implements ResumableUploadCallable.resumeCall(...) to support resuming interrupted resumable upload sessions from a saved session URL and effective chunk size.

@whowes
whowes added this pull request to stack #14678 October 16, 2026 23:06
gemini-code-assist[bot]

This comment was marked as outdated.

@whowes
whowes removed this pull request from stack #14678 October 22, 2026 16:36
@whowes
whowes force-pushed the whowes/resumable-upload-resume branch from 386ec37 to c72e920 Compare October 22, 2026 16:37
@whowes
whowes changed the base branch from whowes/align-chunk-size to whowes/resumable-upload-refactor October 22, 2026 16:39
@whowes
whowes added this pull request to stack #14732 October 22, 2026 16:41
@whowes

whowes commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements session resumption for resumable uploads in the Java GAX library. It introduces the resumeCall method in ResumableUploadCallableImpl, updates ResumableUploadFutureImpl and ResumableUploadProgressTracker to handle resumption, and enhances RewindableStreamBuffer to skip forward to the server's committed offset. Comprehensive unit and integration tests are also added. Feedback suggests preserving the original EOFException as the cause when catching it from ByteStreams.skipFully in RewindableStreamBuffer to maintain the exception chain and avoid potential null messages.

@whowes
whowes force-pushed the whowes/resumable-upload-resume branch 2 times, most recently from c72e920 to cf0c9d4 Compare October 22, 2026 17:20
@whowes
whowes force-pushed the whowes/resumable-upload-resume branch 2 times, most recently from c6f9b2b to b02ed8c Compare October 22, 2026 22:31
@whowes
whowes marked this pull request as ready for review October 22, 2026 22:39
@whowes
whowes requested a review from a team as a code owner October 22, 2026 22:39
@whowes
whowes requested a review from blakeli0 October 22, 2026 22:39
recoveryAlgorithm,
ResumableUploadProgressTracker.createForResume(sessionUrl, options.getChunkSize()));
try {
future.resume(sessionUrl, payloadSupplier);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The future should already have access to both sessionUrl and payloadSupplier, I don't think we have to pass them in. In addition, it seems the main responsibility of future.resume is to create a coordinator, which we may want to inline here.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

queryStatusCallable,
options,
clientContext,
recoveryAlgorithm,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: We may want to refactor the constructor since it has 8 arguments now. For example, I see that recoveryAlgorithm is always created and passed to the coordinator, maybe we can create the recoveryAlgorithm in the coordinator directly.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

setUpGlobalTimeout();
this.payload = checkNotNull(payloadSupplier.get(), "payload supplier returned null");
ResumableUploadChunkCoordinator<ResponseT> coordinator =
createAndRegisterCoordinator(sessionUrl, options.getChunkSize());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there might be a security concern if we use the sessionUrl as it is. If a malicious url being passed in, we would still try to attach auth token to that url and make a call regardless. Is there anyway we can validate the url? Maybe it's host needs to match endpoint from client settings? Also this seems a cross-language concern, do we have any guidance?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Put out a question to the cross-language working group about the general topic. By my reading of the internal docs I suspect that the host of the upload URL isn't guaranteed to be the same as the endpoint where the start command is sent though so I'm not sure what validation is feasible.

Another thought - IIUC the auth headers are only needed for the start command, so perhaps we could unset those in the wire-level callables we use for upload and query.

@whowes
whowes force-pushed the whowes/resumable-upload-resume branch from b02ed8c to 926d744 Compare October 28, 2026 19:32
@whowes
whowes force-pushed the whowes/resumable-upload-resume branch 3 times, most recently from b8d38ad to 5563dbb Compare October 29, 2026 04:56
Base automatically changed from whowes/resumable-upload-refactor to main October 29, 2026 15:36
@whowes
whowes force-pushed the whowes/resumable-upload-resume branch from 5563dbb to a83bf70 Compare October 29, 2026 15:36
@whowes
whowes force-pushed the whowes/resumable-upload-resume branch from a83bf70 to a8d272c Compare October 30, 2026 23:20
@whowes
whowes removed this pull request from stack #14732 October 30, 2026 23:26
@whowes
whowes changed the base branch from main to whowes/resumable-upload-resume-wire October 30, 2026 23:27
@whowes
whowes added this pull request to stack #14871 October 30, 2026 23:27
@sonarqubecloud

Copy link
Copy Markdown

@sonarqubecloud

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed for 'gapic-generator-java-root'

Failed conditions
0.0% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube Cloud

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants